Scalable and Compact 3D Action Recognition with Approximated RBF Kernel Machines

Despite the recent deep learning (DL) revolution, kernel machines still remain powerful methods for action recognition. DL has brought the use of large datasets and this is typically a problem for kernel approaches, which are not scaling up eciently due to kernel Gram matrices. Nevertheless, kernel methods are still attractive and more generally applicable since they can equally manage dierent sizes of the datasets, also in cases where DL techniques show some limitations. This work investigates these issues by proposing an explicit ap- proximated representation that, together with a linear model, is an equivalent, yet scalable, implementation of a kernel machine. Our approximation is directly inspired by the exact feature map that is induced by an RBF Gaussian kernel but, unlike the latter, it is nite dimensional and very compact. We justify the soundness of our idea with a theoretical analysis which proves the unbiasedness of the approximation, and provides a vanishing bound for its variance, which is shown to decrease much rapidly than in alternative methods in the literature. In a broad experimental validation, we assess the superiority of our approximation in terms of 1) ease and speed of training, 2) compactness of the model, and 3) improvements with respect to the state-of-the-art performance

Enhancing visual embeddings through weakly supervised captioning for zero-shot learning

Visual features designed for image classification have shown to be useful in zero-shot learning (ZSL) when generalizing towards classes not seen during training. In this paper, we argue that a more effective way of building visual features for ZSL is to extract them through captioning, in order not just to classify an image but, instead, to describe it. However, modern captioning models rely on a massive level of supervision, e.g up to 15 extended descriptions per instance provided by humans, which is simply not available for ZSL benchmarks. In the latter in fact, the available annotations inform about the presence/absence of attributes within a fixed list only. Worse, attributes are seldom annotated at the image level, but rather, at the class level only: because of this, the annotation cannot be visually grounded. In this paper, we deal with such a weakly supervised regime to train an end-to-end LSTM captioner, whose backbone CNN image encoder can provide better features for ZSL. Our enhancement of visual features, called 'VisEn', is compatible with any generic ZSL method, without requiring changes in its pipeline (a part from adapting hyper-parameters). Experimentally, VisEn is capable of sharply improving recognition performance on unseen classes, as we demonstrate thorough an ablation study which encompasses different ZSL approaches. Further, on the challenging fine-grained CUB dataset, VisEn improves by margin state-of-the-art methods, by using visual descriptors of one order of magnitude smaller

Visualization of Patient Behavior from Natural Language Recommendations

The visualization of procedural knowledge from textual documents using 3D animation may be a way to improve understanding. We are interested in applying this approach to documents relating to patient education for bariatric surgery: a domain with challenging textual documents describing behavior recommendations that contain few procedural steps and leave much commonsense knowledge unspecified. In this work we look at how to automatically capture knowledge from a range of differently phrased recommendations and use that with implicit knowledge about compliance and violation, such that the recommendations can be visualized using 3D animations. Our solution is an end-to-end system that automates this process via: analysis of input recommendations to uncover their conditional structure; the use of commonsense knowledge and deontic logic to generate compliance and violation rules; and mapping of this knowledge to update a default knowledge base, which is used to generate appropriate sequences of visualizations. In this paper we overview this approach and demonstrate its potential

An Interactive Narrative Platform for Story Understanding Experiments

Interactive Narratives are systems that use automated narrative generation techniques to create multiple story variants which can be shown to an audience, as virtual narratives, using cinematic staging techniques. The focus of previous research has included aspects such as the quality of automatically generated narratives and the way in which audiences respond to them. However in this work we have developed a mechanism for control of interactive narratives that supports their use in experiments to assess story understanding. This is implemented in our demonstration system, which features two parts: an interface that allows high-level specification of criteria for story understanding experiments; and a participant interface in which virtual narratives, conforming to the experimental design, are presented as 3D visualizations. The virtual narrative is based on a pre-existing children’s story, and features a cast of virtual characters

Playing in or out of character: User role differences in the experience of Interactive Storytelling

Interactive storytelling (IS) is a promising new entertainment technology synthesizing preauthored narrative with dynamic user interaction. Existing IS prototypes employ different modes to involve users in a story, ranging from individual avatar control to comprehensive control over the virtual environment. The current experiment tested whether different player modes (exerting local vs. global influence) yield different user experiences (e.g., senses of immersion vs. control). A within-subject design involved 34 participants playing the cinematic IS drama "Emo Emma

Two mechanisms drive pronuclear migration in mouse zygotes

A new life begins with the unification of the maternal and paternal chromosomes upon fertilization. The parental chromosomes first become enclosed in two separate pronuclei near the surface of the fertilized egg. The mechanisms that then move the pronuclei inwards for their unification are only poorly understood in mammals. Here, we report two mechanisms that act in concert to unite the parental genomes in fertilized mouse eggs. The male pronucleus assembles within the fertilization cone and is rapidly moved inwards by the flattening cone. Rab11a recruits the actin nucleation factors Spire and Formin-2 into the fertilization cone, where they locally nucleate actin and further accelerate the pronucleus inwards. In parallel, a dynamic network of microtubules assembles that slowly moves the male and female pronuclei towards the cell centre in a dynein-dependent manner. Both mechanisms are partially redundant and act in concert to unite the parental pronuclei in the zygote’s centre

An improved medium formulation for efficient ex vivo gene editing, expansion and engraftment of hematopoietic stem and progenitor cell populations

Gene editing has emerged as a powerful tool for the therapeutic correction of monogenic diseases. CRISPR/Cas9 applied to hematopoietic stem and progenitor cells (HSPCs) has shown great promise in proof-of-principle preclinical studies to treat hematological disorders, and clinical trials using these tools are now underway. Nonetheless, there remain important challenges that need to be addressed, such as the efficiency of targeting primitive, long-term repopulating HSPCs and their in vitro expansion for clinical application. In this study, we assessed the effect of different culture medium compositions on the ability of HSPCs to proliferate and undergo homology directed repair-mediated knock-in of a reporter gene, while preserving their stemness features during ex vivo culture. We demonstrated that by supplementing the culture medium with stem cell agonists and by fine-tuning its cytokine composition it is possible to achieve high levels of gene targeting in long-term repopulating HSPCs both in vitro and in vivo, with a beneficial balance between preservation of stemness and cell expansion. Overall, the implementation of this optimized ex vivo HSPC culture protocol can improve the efficacy, feasibility and applicability of gene editing as a key step to unlocking the full therapeutic potential of this powerful technology

Interacting with Virtual Characters in Interactive Storytelling. In:

ABSTRACT In recent years, several paradigms have emerged for interactive storytelling. In character-based storytelling, plot generation is based on the behaviour of autonomous characters. In this paper, we describe user interaction in a fully-implemented prototype of an interactive storytelling system. We describe the planning techniques used to control autonomous characters, which derive from HTN planning. The hierarchical task network representing a characters' potential behaviour constitute a target for user intervention, both in terms of narrative goals and in terms of physical actions carried out on stage. We introduce two different mechanisms for user interaction: direct physical interaction with virtual objects and interaction with synthetic characters through speech understanding. Physical intervention exists for the user in on-stage interaction through an invisible avatar: this enables him to remove or displace objects of narrative significance that are resources for character's actions, thus causing these actions to fail. Through linguistic intervention, the user can influence the autonomous characters in various ways, by providing them with information that will solve some of their narrative goals, instructing them to take direct action, or giving advice on the most appropriate behaviour. We illustrate these functionalities with examples of system-generated behaviour and conclude with a discussion of scalability issues